The present invention relates generally to real-time event processing, and more particularly to methods, apparatus and systems for processing real-time events in applications such as telecommunications and computer networks.
Many event-processing systems have real-time performance requirements that cannot be met by conventional general-purpose database management systems. In telecommunications applications, for example, a variety of adjunct switching services such as debit-based billing, number mapping, call forwarding, and local-number portability involve event processing during the critical call-connection phase of a telephone call. To meet the real-time requirements of the network, the service time for such events generally must not exceed a few milliseconds. However, with conventional database technology, the costs of invoking a structured query language (SQL) operation over a client-server interface, or the costs associated with a single access to secondary storage, can account for hundreds of milliseconds. As a consequence, performance goals on the order of a few milliseconds may already be unattainable even before the costs of the event processing logic are taken into account.
These limitations have led to the use of custom database systems for many high-performance real-time event processing applications. Custom systems are typically tightly coupled to their particular applications, and are tuned to the specific requirements of those applications. These solutions generally work well in practice, and can provide close to optimal performance. However, they also have a number of serious disadvantages. For example, the cost of developing and maintaining custom systems can be high, and generally cannot be amortized across a number of different applications. Moreover, custom systems are frequently inflexible. It can become difficult or even impossible to adapt a custom system to unforeseen or evolving requirements.
A need therefore exists for an improved real-time event processing system which provides the performance benefits of custom database systems, without sacrificing the flexibility and maintainability typically associated with conventional general-purpose database systems.
The invention provides a general-purpose real-time event processing system (EPS) which avoids the problems associated with custom systems. The EPS in an illustrative embodiment meets real-time performance goals through the use of one or more real-time analysis engines (RAEs) operating in conjunction with a main-memory storage manager as its underlying database system. The main-memory storage manager offers transactional access to persistent data, but at the speed of a main-memory system. Application-specific services for execution in the RAEs of the EPS are generated in a service authoring environment (SAE). The SAE provides a set of tools and one or more graphical user interfaces (GUIs) for authoring, validating, compiling and installing new services, or adapting existing services to new requirements.
In accordance with the invention, the SAE which is used to generate services executed in the RAEs may include a declarative language comprising one or more of the following data structures: (i) a base table which supports insert, update, delete and select operators, (ii) a chronicle which supports insert and select operators, wherein an insert into a chronicle corresponds to generation of a record on an output stream, and (iii) a view which supports a select operator and whose contents are derived from one or more other tables. Both the base tables and the views may be stored explicitly in a memory portion of the main-memory database system accessible to the RAEs, but chronicles are generally not stored in the memory portion of the main-memory database system. The SAE further includes a GUI for generating the services, a static check element for processing a given service generated by the GUI, and a translation and compilation element for generating program code for implementing the given service within the RAEs. The SAE may generate at least one of the services in the form of object code which is dynamically linked into the RAEs without interrupting event processing.
A real-time EPS in accordance with the invention provides a critical path for event processing that is specifically designed for high performance, while also retaining many desirable features of conventional database systems, including high-level, declarative programming interfaces, and the well-known transactional correctness properties of atomicity, consistency, isolation and durability (ACID). These features of the invention enhance the reliability, robustness, usability and maintainability of the real-time EPS and the applications built thereon.
The invention can be used in conjunction with any event processing application, including, for example, telecommunications, electronic commerce, and Internet service provisioning applications. For example, in a telecommunications application, the invention can be configured to provide a basis for features such as enhanced billing systems, fraud detection and prevention, local-number portability, settlements among service providers and real-time traffic analysis.